Estimation is Easy and Useful: Estimate a game of Chess
Estimating Software Development
Anyone who knows me will know that the title of this post is meant as a bit of a joke. For software development, it is my experience that estimating** is rarely useful and always easy.
Well… a little clarification: estimating is NEVER useful but ALWAYS easy the way I have typically seen it done, which uses the WAG* method of providing some numbers that seem plausible and that will allow the decision makers (stakeholders and managers) to approve the start or continuance of a project with a clear conscience. That is, when things sour later on they can always blame the developers for giving them inaccurate estimates (and not living up to their “commitments”).
And the developers can always blame the “requirements people” for giving them unclear, incorrect, or incomplete requirements.
And the requirements people can always blame the stakeholders for providing bad direction about what was important and the developers for not “understanding the requirements”, and so on.
Regardless of who is part of this process, it is one big happy circle of blame that lets us all do the wrong thing and still keep our jobs (not always happy for everyone, actually, and sometimes not everyone will keep their job).
I have simplified it here, but the basic idea is sound. As long as everyone is good at deflecting blame, and everyone is willing to continue on saying we’ll do it better “next-time” and after only a few people have been fired then everything is fine, right? Well, I don’t think it’s fine, of course – but many organizations seem to operate this way.
Let’s estimate a game of chess
Chess is a game that some of you may have seen. It provides a very limited environment with only 6 different and charming pieces that have very limited ways in which they can be used, and there are only two players. Simple. The whole game can be contained in an area of about a square foot or even less, and the complete rules can be written on the inside of the box lid. Just like Candy Land, it is very easy to learn and play. Not so easy to win, perhaps – but that makes it interesting enough to be an engaging game for some.
So, here is a description of a simple “project” and I need your estimate:
Win a game of chess. Losing is not an option. Tying is not an option. We will only make money if you win. If you don’t win, the company is at risk, and you will lose your job (and mine).
Also, I have a diagram of a chess game with the exact position of the pieces as required at the end of the game. To make money, it needs to end up just like the diagram.
Oh, one other thing: we don’t know yet who you will playing, but we’re pretty certain it will be someone who knows how to play the game, and might be pretty good at it.
That’s it! There are only a few requirements: That you win, and that the board matches the diagram I’ll give you. Easy!
What I need to know is exactly how much this will cost me. In this situation, cost is the number of moves you’ll need to meet the requirements. Oh, also, we don’t have the diagram yet, but we’ll have it soon. Still, you should be able to estimate it because it is basically the normal chess end-of-the-game arrangement with pieces here and there and so on, and you have the opponent in a checkmate.
Should be easy to estimate that, right?
Well, I hope you agree this is impossibly difficult, or actually: just plain impossible to do a meaningful estimate. First of all, there is not enough information to do a good estimate, and even if we had all the information, there are many variables and a lot of stuff we have no control over. How close do you think your end results would come to the requirements? Would you be willing to stake your job on making a decent estimate on a chess game in this situation? Well, as long as you could blame someone else and promise to “all do better next time” you’ll probably be okay. Or you could be a consultant and be paid and off to your next contract before someone actually has to play this game you helped them estimate.
Getting better at estimating is like pulling teeth
I hear it over and over: Our estimates were not accurate and we had a lot of trouble because of that. We need to get better at estimating.
It is similar in a way to saying “My tooth hurt, so we pulled it out. Now I can’t chew as well as I used to. We need to get better at pulling teeth”. What is the real problem? Shouldn’t we work on that instead? Hint: Getting better at pulling teeth is not it.
So what would you do to get better at estimating the game of chess, based on the basic scenario we described above? Could you get better at it? How good would you have to be before it had real value? Is it worth spending time on getting better at it? What is the real problem. Hint: Getting better at estimates is not a solution to anything because having bad estimates it not the real problem.
My suggestion: Stop That!
I haven’t seen much value in estimates. Actually, in the software development work I have done, I can’t recall any situation where estimates** of this sort were of value to the actual job at hand. Someone wanted them, someone saw value in them, and sadly – important decisions were made based on them. Do you think those were good decisions? People often tell me that “even though we know the estimates are inaccurate, they are better than nothing”. I prefer nothing, if there are better ways. There are better ways.
Almost everyone tells me that estimates** are very important, but estimates are often a blind spot hiding something of actual importance and value. This almost universal acceptance that estimates are important and useful is harmful. A predictive approach such as waterfall itself depends on estimates – and estimates are how we make the predictions. Don’t think “waterfall” and predictions are harmful? You probably should be reading some other blog.
Estimates** of this sort are based on guesses about the time needed to do work we don’t yet understand. Nothing about this gives me confidence that these estimates have value.
It seems I always make enemies when I suggest this, but that is not my intention. I want to do effective work on meaningful stuff – that is my intention. I don’t want to be spending time on work that has no value, causes misdirection and waste, and potentially destroys any chance of being effective. That is painful. There are MUCH BETTER WAYS to approach software development.
Do you need estimates?
It could be useful for you and your organization to question your use of estimates and explore new ways to think about your “need” for estimates.
One last thing: Do not ask me for an estimate on how long it will take to eliminate estimates from your process. It is too much like a game of chess, only very complicated.
Here is a start at clearing up our thinking. Let’s do a little thought experiment.
Which would better help if we have decided to eliminate estimates and find a better way in our organization? :
- Get rid of all developers
- Get rid of all upper management.
- Get rid of all upper management and developers
- Something else more realistic
Who is most insistent that estimates are important? Who do those people answer to? Are they going to fire you for questioning things like estimates? Are they open to scrutinizing the status quo or to examining why we do the things we do?
So what might be better?
What can we do instead of doing estimates?
Nothing. Don’t do estimates. Simple, clean, easy. But that is not the question we should be asking, and that is not what I am suggesting.
The correct question is: since we think that estimates are important, and you say that they are not, how can we develop software without doing estimates?
The answer is… Well, I suspect no one reads this blog, so that doesn’t really matter. This is just a clearing house for my thinking and has nothing to do with the reader. However, I am sure to write some stuff someday about ways to get things done without estimates, and I guess some of my earlier posts cover some of this thinking. If someone is actually interested and asks for me to cover this, I might get to it sooner. But I am sure you can figure it out without me – this is truly simple stuff. Read the Agile Manifesto and Agile Principles, and use them to invent better ways.
Another hint: Get good at frequent incremental delivery of useful software. That might be worth considering. That takes a LOT of experience and open thinking.
NOTES:
* WAG = Wild Ass Guess. A Wild Ass is a beast that is indigenous to parts of Africa and Asia. A Guess is a guess. Wild asses are about as good at making software estimates as anyone else.
** For the purpose of this article, the sort of estimates I am discussing are the estimates typically asked for on many projects where a list of features or functions are described and people are asked to come up with an approximate cost in time (working time or elapsed time) [sometimes it is phrased as “cost”, or “effort”] to do the work that will be required to provide the feature(s)/function(s)/capability(ies) being requested.
Disclaimer: There are many situations where estimates can be meaningful and useful. This article is about situations where I don’t think they are meaningful or useful.
Aaron Griffith:
Reading this made me realize I used to get frustrated with upper management for not following agile practices. Now I get frustrated because they claim they are agile but are really just screwing it up and giving it a bad name.
7 November 2011, 1:47 pmWoody Z.:
Hey Aaron. I think that the Agile namespace has been poluted. Not too many places claiming to be “Agile” or trying to follow an agile methodology are actually “doing Agile”. In most cases this is unintentional, but is a sign of how difficult it is to manage software development. Most folks are just trying to get on with their work, and are hoping the latest “new thing” (Agile, in this case) will go away so things will settle back into the same old “same old” where they get to keep plodding through the work-week without interuption. Okay – I’ll try to keep from being too depressing – just keep saying “This too shall pass, this too shall pass”
7 November 2011, 2:03 pmDavid A:
Yes, I’d be interested in how you can get things done without (any) estimates.
Or more importantly, how you can get things *started*. How can you get anyone to commit money to a project if you have no estimate of whether the goal is even remotely possible given the available budget?
It’s not such a problem for incremental changes to an existing system, but if you want an entirely new capability, then there’s often a minimum spend before you get anything of genuine value. You need to know if you can afford at least that minimum spend. Getting 90% of the way there is not an option.
You could (in theory) just get started and see how things go… but without estimates, how do you know if you can safely extrapolate the progress so far? Given fine-grained features, one could perhaps assume that they are all the same size, and use that as a basis, but you can’t sanity-check that assumption without at least some order-of-magnitude estimation (how do you know that your features are, in fact, fine-grained?).
7 December 2012, 11:57 amWoody Z.:
Hello David. Thanks so much for asking. I will start posting more about this a little at a time. I think it is simple, but every time I write about it the words mount up quickly. So… I’m breaking it down into small ideas about the approaches I have taken and the motivations behind them. But again… Thanks for asking!
7 December 2012, 1:31 pmCory House:
I can appreciate the feeling of fruitlessness in some estimates. It’s hard to get right, particularly when requirements are weak. However, I believe a professional developer must be willing to take the risk and do the hard work of estimating when requested. Drawing a hard line of “no estimating” without considering context doesn’t come across professional or accommodating.
Your chess example is entertaining, but it’s also a great example of a place where I feel it’s unfair to expect an estimate. Given, it’s gray exactly how your story would translate to the development world, but it seems to describe a request to estimate a project without even vaguely reasonable requirements. I think we all agree that’s a waste of time.
If given sufficient requirements, we, as professionals, should be able to estimate our work. The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.
I land plenty of dev work on contract because I’ve gotten good at asking the right questions and sniffing out complexity so that I can provide reasonable estimates. Being skilled in estimation sets you apart whether you’re independent or full-time. Steve McConnell wrote an excellent book on the subject. http://www.amazon.com/Software-Estimation-Demystifying-Practices-Microsoft/dp/0735605351
That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.
10 January 2013, 6:34 pmWoody Z.:
Hello Cory! Thanks for commenting on my post!
Well, as you can guess, I don’t agree with your take on things – but that’s why I have a blog, and you have a blog. We all get to write about things as we see them.
Regarding professionalism: I believe it is professional to find a better way to do all the things we do, and eliminate things that are wasteful. I see estimates and the estimating process as waste, and as a professional I see no need to “take on the hard work of estimating when requested”. More directly: estimating is not hard to do, in my opinion. I’m not clear on why you might think it is. However, I also believe that estimates as used in software development typically (and almost always) result in misinformation that is used to make bad decisions. Which is why I follow an Agile approach to development. The Agile approach, at least as I do it, provides a lot of value and eliminates a lot of waste. And that is very professional. We’ll just have to agree to disagree on our thinking about professionalism and estimates.
I don’t think I made it clear in the post: The Chess analogy is to show that something as simple as Chess becomes complicated quickly. It is a VERY SIMPLE GAME, with very few variables, and yet it becomes too difficult to predict when we add a few simple constraints. And even without those constraints, there is little we can predict. Otherwise we wouldn’t even play the game, right? Remember: Don’t stretch an analogy too far – it will always break. With software projects, anything that is non-trivial is much more complicated than the simple chess example.
You: “If given sufficient requirements, we, as professionals, should be able to estimate our work”.
My Response: This is where we really diverge. I see estimates as waste, in the Lean sense of the word, and every other way. Estimates are Muda. They are not part of what the customer buys. They don’t add value to the product. In a value stream map estimates are on the “non-value” track. They are merely ONE WAY people have attempted to be able to arrange working relationships and get software created. It makes sense, and it partly works – still, there is nothing about them that makes them necessary, or even beneficial to the actual creation of software. We can write software without them. We can do whole projects without them. And if we can find a way to make BETTER SOFTWARE at less cost, and to more quickly get the right software into use without estimates, wouldn’t it be professional and worthwhile to at least consider doing it that way? Hopefully we can agree on that.
You: “The world expects no less from engineers in other fields. Multiple contractors recently estimated the cost to finish my basement despite the fact that I had no floor plan, no idea where lights would be, only a vague idea on what word work I’d like, a bathroom of unknown size and quality, etc. If one of them refused to provide an estimate, I’d have gone elsewhere.”
My Response: Here we continue to think differently. There are countless examples in life where estimates work to some degree. We’ll discuss that some other day – most, if not all of them are not useful as models for software development. Here is where I’d like to leave it on this for the moment: If you could have had the work done on your basement for less money and get higher quality, and perhaps get it done even quicker – would you be willing to consider doing that? I know that there would be no way to know it was cheaper if you had no estimates – but conceptually: If you could get all the benefits I describe would you consider learning to do business that way? I hope we can agree that would be better all other things being equal.
We have contracts, and estimates, and predictions, and all the trappings of doing business this way because most people haven’t found a way to do things better – we fail each other, we don’t deliver as expected, we don’t trust each other, so we find ways to do business that gives us a chance to do business. So… we can try to get good at estimates, and it if serves you and your customers, more power to you.
Lastly:
You: “That said, while I disagree with your answer to “What can we do?” as “Nothing.”, I enjoyed reading your blog. It’s fun reading and certainly food for thought.”
My Response: I didn’t ask “What can we do?”. I asked “What can we do instead of doing estimates?”, and the answer was simply put as “Nothing”. My point being that we do not need to do anything “instead” – estimates are UNNEEDED. And then I posed the more meaningful question: “how can we develop software without doing estimates?”. The nuances of this are important, and perhaps I was being too cryptic. At the time I wrote this post it seemed few were willing to explore these things. I hope I was wrong, and now I am certainly finding more people willing to entertain and even experiment with finding better ways. And some doing purely “no estimate” projects.
Overall, I have found the Agile way works well. When you have customers who see the benefits and find it works better for them than the old phased, predictive approach – then it is truly sweet.
Cheers!
10 January 2013, 8:08 pmLife, Liberty, and the Pursuit of Agility » Blog Archive » A Comment And Response from Estimate Chess Post:
[…] House ( blog , twitter) wrote a comment on my Estimate Chess post, and I’d like to double purpose it by turning it into a […]
10 January 2013, 8:54 pmAndrew Fox:
I completely agree in principal that estimation is waste. I’ve used both a form of Scrum and a form of Kanban in the past and as a practitioner I much prefer the continuous flow of Kanban because it doesn’t require the downing of tools, wiping board clean and estimation or re-estimation (all waste) that a lot of people do with Scrum. In continuous projects that have a clearly defined framework in which to work, I think this can work fairly well.
So I guess in this respect I agree with you when you suggest “don’t estimate”. If you shorten cycles enough, you discover and plan as you go which reduces the need for estimates.
Saying that, the reality is that when there are many unknowns maybe on a new project, the likes of stakeholders and product owners are looking at the project as a business case and feel the need for estimates. They see it as one giant thing that they want delivered and like you say, teams estimate to mitigate risk and so they can play a blame game against other teams.
I look ahead to a future when organisations take a more lean approach to new product development. Instead of “We want a big new thing we’ve just imagined and we want it now” it’ll be more “Lets try this out, it may work it may not but let’s do the minimum we can to work it out and learn how to add value as we go”. I think “No estimation” works hand in hand with this approach but obviously it requires a bigger mindshift.
14 January 2013, 11:16 amWoody Z.:
Hello Andrew – thanks for your comment!
I agree wholeheartedly – the stakeholders/product owners/customers are a big part of this, and it is for the benefit of the company and the stakeholders/customer/product owner and the product itself that we need to find a better way than estimates to make the decisions about “which/when/how” etc.
We all need to work together to find better ways. There ARE companies/teams discovering and using these ideas – and the rest will find themselves falling behind competitively if they don’t at least explore these things.
Cheers!!!
14 January 2013, 12:09 pmFred Z.:
I agree that estimates are easy and useless. We do a form of “agile” process where I currently work. We always estimate stories using a very easy WAG process. Not much weight is put on the meaning, and it doesn’t take much time. It’s a good group to work for. I participate in the estimate process by throwing out my “2” or “3” or “5” when asked because I don’t want people to think I’m as rude and stubborn as you, even though I am. When we need a tie-breaker and I’m asked to defend my estimate, I often say “because my birthday is the 3rd” or “because 5 seems more fun”. Often I’ll defer my vote to a team member “who actually knows what they’re doing”
1 February 2013, 9:10 amFred Z.:
Meant to say “It’s a GOOD group to work for”
[Fred: I edited that for you]
1 February 2013, 9:12 am